Parallelize #1960

kderme · 2025-03-21T12:41:53Z

Description

TODO:

the generated tx.id are generated as block.id * 100000 + tx_index. The 100000 is chosen to avoid duplications, since the genesis virtual block has ~14000 txs, . A cache could be used to get the last id for each block.
Commiting the main block transaction should first wait for all dependencies transactions to be commited
Check db to avoid reinstering rewards if there is a restart.
soptEpochAndCacheEnabled is hardcoded False. Fix it.
forcing the rewards from the ledger state needs major simplification and may be bugged (see genericRewardUpdate). Currently the ledger functions that compute them live in ReaderT Global even though the Globals are never used. This could be improved in the ledger api, by providing something similar to forceDRepPulsingState
Remove reduntant Promise.hs and TxInResolve.hs
Threads should not commit if nothing happened since the last commit. Otherwise warning are raised.

Further improvements:

In function applyAndInsertBlocks we split the blocks based on byron or not. We should also split them for epochboundaries. To achieve this, we need to predict the slot details of the whole list of blocks, before the ledger application is done. This is possible since we can predict the slot details up to a specific horizon
Use the Prepared tx type, so that all preparations happen in parallel.

Checklist

Commit sequence broadly makes sense
Commits have useful messages
New tests are added if needed and existing tests are updated
Any changes are noted in the changelog
Code is formatted with fourmolu on version 0.10.1.0 (which can be run with scripts/fourmolize.sh)
Self-reviewed the diff

Migrations

The pr causes a breaking change of type a,b or c
If there is a breaking change, the pr includes a database migration and/or a fix process for old values, so that upgrade is possible
Resyncing and running the migrations provided will result in the same database semantically

If there is a breaking change, especially a big one, please add a justification here. Please elaborate
more what the migration achieves, what it cannot achieve or why a migration is not possible.

Also move some util functions

TODO: this needs simplification

Not done yet for Address variant

Cmdv

looking very good so far, just very minor comments

Cmdv · 2025-05-02T07:42:20Z

cardano-db-sync/src/Cardano/DbSync.hs

@@ -237,6 +243,7 @@ extractSyncOptions snp aop snc =
        not isTxOutConsumedBootstrap'
          && ioInOut iopts
          && not (enpEpochDisabled snp || not (enpHasCache snp))
+          && False


I assume this is for testing

Yes I have it in the TODO list of the pr

Cmdv · 2025-05-02T07:48:19Z

cardano-db-sync/src/Cardano/DbSync/Api.hs

@@ -390,10 +346,59 @@ mkSyncEnv trce backend connectionString syncOptions protoInfo nw nwMagic systemS
  where
    hasLedger' = hasLedger . sioLedger . dncInsertOptions
    isTxOutConsumedBootstrap' = isTxOutConsumedBootstrap . sioTxOut . dncInsertOptions
+    backend = mainBackend backends
+
+withDBSyncConnections ::


Should these maybe not be inside of Cardano.Db.Run

Yes it could be. I thought DbConnections as something that DBSync maintains and could change based on its needs, but this is debatable

Cmdv · 2025-05-02T07:50:39Z

cardano-db-sync/src/Cardano/DbSync/Api/Types.hs

@@ -100,3 +114,22 @@ data ConsistentLevel = Consistent | DBAheadOfLedger | Unchecked
 newtype CurrentEpochNo = CurrentEpochNo
  { cenEpochNo :: Strict.Maybe EpochNo
  }
+
+data DbConnections = DbConnections


We'll have conflicts with these as hasql doesn't use these types :)

Cmdv · 2025-05-02T07:54:46Z

cardano-db-sync/src/Cardano/DbSync/Cache.hs

 import Cardano.DbSync.Cache.Epoch (rollbackMapEpochInCache)
 import qualified Cardano.DbSync.Cache.FIFO as FIFO
 import qualified Cardano.DbSync.Cache.LRU as LRU
-import Cardano.DbSync.Cache.Types (CacheAction (..), CacheInternal (..), CacheStatistics (..), CacheStatus (..), StakeCache (..), initCacheStatistics, shouldCache)
+import Cardano.DbSync.Cache.Stake as X
+import Cardano.DbSync.Cache.Types


should we also expose these modules same a Cardano.DbSync.Cache.Stake?

Stake is the only part of Cache which is splitted in its own module.
If you're refering to Cardano.DbSync.Cache.Types, we don't usually reexpose the Types through its most related module. eg see Cardano.DbSync.Api.(Types)

Cmdv · 2025-05-02T08:01:44Z

cardano-db-sync/src/Cardano/DbSync/Cache/Stake.hs

+import qualified Data.Map.Strict as Map
+import Database.Persist.Postgresql (SqlBackend)
+
+-- | TO be called only by the stake thread


If you fancied it you could have a type level constraint on thread specific functions:

-- Define thread-specific phantom types data StakeThread data BlockThread data TxThread -- Create a type that represents actions constrained to specific threads newtype ThreadConstrained thread m a = ThreadConstrained { runThreadConstrained :: m a } deriving (Functor, Applicative, Monad, MonadIO, MonadBaseControl IO) -- Modify your function to be constrained to the stake thread resolveInsertRewardAccount :: forall m. (MonadBaseControl IO m, MonadIO m) => SyncEnv -> CacheAction -> RewAccount -> ThreadConstrained StakeThread (ReaderT SqlBackend m) DB.StakeAddressId resolveInsertRewardAccount syncEnv cacheUA ra = ThreadConstrained $ do eiStakeId <- queryStakeAddrWithCacheRetBs syncEnv cacheUA False ra -- read only case eiStakeId of Right stakeId -> pure stakeId Left (_, bs) -> insertStakeAddress ra (Just bs) -- Then in your stake thread code, you would run it like: runStakeThread :: MonadIO m => ReaderT SqlBackend m () runStakeThread = do -- Get your parameters from somewhere let syncEnv = ... cacheUA = ... ra = ... -- Run the thread-constrained function stakeId <- runThreadConstrained (resolveInsertRewardAccount syncEnv cacheUA ra) -- Do something with stakeId ... -- If someone tries to call it from a different thread, they'd need to write: runBlockThread :: MonadIO m => ReaderT SqlBackend m () runBlockThread = do -- This won't type-check because BlockThread ≠ StakeThread stakeId <- runThreadConstrained (resolveInsertRewardAccount syncEnv cacheUA ra) ...

Could be yes. Once this is settled, it can become more type safe.

Cmdv · 2025-05-02T08:10:53Z

cardano-db-sync/src/Cardano/DbSync/Era/Shelley/Genesis.hs

  where
+    txId = DB.TxKey $ fromIntegral txIndex + 20000 * DB.unBlockKey blkId -- On mainnet genesis produces ~14000 txs


maybe a little comment as to why we're not using the returned txId from running insertTx. I can't fully remember why :)

Personal note to add

We're using app generated unique ids to avoid dependencies between tables, which can damage parallelism.

cardano-db-sync/src/Cardano/DbSync/Ledger/Async.hs

kderme added 4 commits March 18, 2025 20:52

Add Epoch Stake Thread

cc6ffbc

Add Stake thread

e7e0f4b

Regroup modules

ed8a586

Add Resolve Inputs thread

064aaf1

kderme force-pushed the kderme/parallelize branch from c024537 to 064aaf1 Compare March 24, 2025 10:43

kderme added 4 commits April 4, 2025 14:09

Start using explocit Ids for Tx and Block

256d0f5

Parallelise preparing the txs

6bc6098

Also move some util functions

Parallelize ledger application

02aaaca

Force calculation of rewards

c38499b

TODO: this needs simplification

kderme force-pushed the kderme/parallelize branch from ff015c7 to a6f18a4 Compare April 24, 2025 12:41

Add rewards thread

7209d87

kderme force-pushed the kderme/parallelize branch from a6f18a4 to 7209d87 Compare April 25, 2025 10:07

kderme added 3 commits April 26, 2025 21:31

Fixes

6de7ea8

Fix generated txIds

04576b6

Add MA thread and dedicated connections for datum and scripts

3373fee

Not done yet for Address variant

kderme force-pushed the kderme/parallelize branch from 92c9850 to 3373fee Compare April 30, 2025 20:56

Commit secondary transactions before the main one

b4934be

kderme force-pushed the kderme/parallelize branch from 898c1d1 to b4934be Compare April 30, 2025 22:15

Cmdv reviewed May 2, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parallelize #1960

Parallelize #1960

kderme commented Mar 21, 2025 •

edited

Loading

Cmdv left a comment

Cmdv May 2, 2025

kderme May 2, 2025

Cmdv May 2, 2025

kderme May 2, 2025

Cmdv May 2, 2025

Cmdv May 2, 2025

kderme May 2, 2025

Cmdv May 2, 2025

kderme May 2, 2025

Cmdv May 2, 2025

kderme May 2, 2025

		where
		txId = DB.TxKey $ fromIntegral txIndex + 20000 * DB.unBlockKey blkId -- On mainnet genesis produces ~14000 txs

Parallelize #1960

Are you sure you want to change the base?

Parallelize #1960

Conversation

kderme commented Mar 21, 2025 • edited Loading

Description

Checklist

Migrations

Cmdv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kderme commented Mar 21, 2025 •

edited

Loading